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1  Introduction 

This  is  the  System  Requirements  /  Concept  Report  (CDRL  Data  Item  #A003)  for  EAITS,  an 
extensible  and  adaptive  STEM  Learning  System  that  BBN  is  developing  under  the  ONR  STEM  Grand 
Challenge  (GC)  Program,  Contract  #  N00014-12-C-0535.  EAITS  will  embody  algorithmic  research  and 
system  development  for  high-school  and  college  level  advanced  learning  systems.  These  learning 
systems  will  employ  state-of-the-art  pedagogical  and  computation  techniques  to  improve  student 
proficiency  in  targeted  subject  areas. 

For  the  ONR  STEM  GC  Program,  BBN  will  develop  a  prototype  version  of  EAITS  and  demonstrate 
its  customization  to  the  SAT  Physics  curriculum.  In  this  document,  we  first  will  present  a  conceptual 
overview  describing  the  principles  and  plans  for  EAITS  development  and  evaluation.  The  second  part  of 
this  document  specifies  the  software  requirements  that  will  be  met  by  the  SAT  Physics  version  of  EAITS. 


2  Conceptual  Overview 

The  EAITS  will  advance  the  state-of-the-art  in  learning  technologies  through  the  fusion  of 
multiple  forms  of  adaptive  support  with  highly  effective  learning  activities.  Each  of  these  forms  of 
support  and  activities  have  been  individually  shown  to  be  effectively  in  terms  of  improving  learning 
outcomes  in  prior  work.  Through  meticulous  software  integration  and  careful  coordination  achieved  by 
statistical  models,  we  can  close  the  gap  between  the  efficacy  of  automated  learning  systems  that  are 
scalable  for  mass  use  and  the  2 a  efficacy  of  scarcely  available  human  tutors. 

The  SAT  Physics  version  of  EAITS  will  be  an  advanced  learning  system  that  allows  high-school 
students  to  improve  their  proficiency  at  solving  SAT-style  Physics  problems.  The  system  achieves  this 
through  a  two  learning  activities.  First,  EAITS  will  allow  students  to  engage  in  learning  through  problem 
solving.  Second,  students  can  explore  multimedia  instructional  content  such  as  videos  about  Physics 
concepts.  The  student  can  also  participate  in  both  of  these  learning  activities  in  a  self-paced,  anytime, 
anywhere  manner  using  a  variety  of  devices  and  platforms  that  have  a  standards  compatible  web- 
browser. 

The  learning  efficacy  of  this  system  will  be  amplified  through  the  use  of  personalized  adaptive 
tutoring  that  in  integrated  in  both  the  learning  activities  described  above.  The  tutor  will  use 
comprehensive  student  modeling  in  combination  with  a  domain  model  and  rigorous  analysis  of  the 
student's  actions  in  the  learning  environment  to  constantly  update  and  adapt  its  beliefs  about 
proficiencies  and  preferences  of  the  student.  For  example,  if  a  student  demonstrates  the  inability  to 
solve  a  problem  involving  the  concept  of  resistivity,  the  tutor  updates  it  belief  about  the  student's 
proficiency  in  this  concept  and  may  decide  to  initiate  interventions  that  will  lead  to  the  improvement  in 
this  the  student's  knowledge.  The  EAITS  tutor  will  be  capable  of  using  multiple  forms  of  support  to  tailor 
the  interventions  delivered  to  the  student  including  the  use  of  corrective,  reflective,  and  engagement 
support.  Furthermore,  the  intervention  presented  to  the  student  will  be  informed  by  prior  observations 
(captured  in  the  student  model)  such  as  the  belief  that  a  natural  language  dialog  may  be  more  effective 
for  certain  learners  than  the  use  of  instructional  videos. 
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The  tutor  will  also  use  data-driven  assessment  model  to  estimate  and  predict  the  expected 
performance  of  a  student  on  standardized  tests  within  the  topics  covered  by  the  system.  This 
assessment  capability  will  serve  as  a  feedback  mechanism  that  regulates  the  balance  between  tutor  and 
student  initiative  during  learning  activities. 

In  addition  to  the  SAT  Physics  version  of  the  EAITS,  the  underlying  platform  will  provide 
authoring  tools  that  support  development  and  maintenance  of  high-quality  learning  content  that  can  be 
used  within  the  EAITS.  These  tools  can  be  used  by  domain  experts  and  system  administrators  to  add  and 
modify  knowledge  sources  used  by  SAT  Physics  EAITS  in  an  ongoing  manner  without  the  need  to  modify 
the  underlying  software  platform. 


3  Requirements 

This  report  is  a  living  document  that  captures  on-going  requirements  discussions  between  ONR 
and  BBN.  Early  versions  will  focus  on  broad  requirements  such  as  generic  system  functionality.  Later 
versions  will  refine  the  requirements  to  include  specific  operating  characteristics  such  as  deployment 
specifications. 

The  following  sections  are  organized  by  requirement  categories.  Sections  itemize  specific 
requirements,  each  of  which  have  a  unique  identifier  in  support  of  traceability  for  design,  testing,  and 
validation  plans. 


3.1  Capability  Requirements 

Req.  1.  EAITS  shall  allow  students  to  learn  about  SAT-level  Physics  for  the  following  topics: 

Req.  l.a  Electromagnetism 

Req.  2.  Learning  in  EAITS  shall  be  enabled  through  multiple  learning  activities  including 

Req.  2. a  Structured  exploration  of  rich  instructional  content  integrated  within  the 
learning  environment  including 
Req.  2.a(i)  Text 

Req.  2.a(ii)  Videos 

Req.  2.a(iii)  Natural  Language  Dialogs 

Req.  2.b  Interactive  problem  solving  covering  the  complete  range  of  SAT  Physics  problem 
types  including 
Req.  2.b(i)  Verbal 

Req.  2.b(ii)  Pictorial/Spatial 

Req.  2.b(iii)  Graphical 

Req.  2.b(iv)  Symbolic 

Req.  2.b(v)  Numerical 

Req.  2.b(vi)  Recall 

Req.  2.b(vii)  Single-Concept  Problem 

Req.  2.b(viii)  Multiple-Concept-Problem 

Req.  3.  EAITS  shall  provide  multiple  forms  of  automated,  personalized  support  to  the  learning 
activities  listed  in  Req  2.b  including 

Req.  3. a  Corrective  support  through  use  of  problem-specific  tutor  models 
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Req.  3.b  Reflective  support  through  integration  of  tutorial  dialog  with  problems 
Req.  3.c  Engagement  support  through  the  use  of  timely  social  interaction  strategies 

Req.  4.  EAITS  shall  provide  reporting  about  the  learners  that  includes 

Req.  4.a  Quantitative  proficiency  assessment  through  the  use  of  automatic  assessment 
algorithms  built  on  student  modeling  and  machine  learning  algorithms 
Req.  4.b  An  student  profile  that  visually  summarizes  the  learning  trajectories,  skills,  and 
achievements  of  each  student 

Req.  5.  EAITS  shall  provide  an  authoring  suite  that  can  be  used  by  content  developers  (such  as 
educators  and  domain  experts)  to  maintain  and  extend  the  delivered  EAITS.  These  authoring 
tools  will  include  the  following  integrated  tools 
Req.  5. a  Domain/curriculum  authoring  tool 

Req.  5.b  Problem  and  tutoring  support  authoring  tool 
Req.  5.c  Tutorial  dialog  authoring  tool 

Req.  6.  EAITS  shall  incorporate  a  data  repository  that  captures  and  stores  time-stamped  events  logs 
generated  through  the  use  of  the  EAITS 

3.2  Design  Requirements 

Req.  7.  EAITS  separate  content  and  code  to  allow  extension  of  the  system  in  terms  of 
Req.  7. a  Inclusion  of  additional  topics  in  the  SAT  Physics  curriculum 

Req.  7.b  System  tailoring  to  support  other  STEM  learning  domains 

Req.  8.  EAITS  shall  be  modular  and  flexible  to  allow  integration  of  new  algorithms  and  models, 
specifically  for 

Req.  8.a  Tutor/support  modeling 

Req.  8.b  Student  modeling 

Req.  8.c  Automated  assessment 

Req.  9.  The  EAITS  environment  used  by  students  to  access  the  learning  activities  shall  be  cross¬ 
platform  compatible  through  the  use  of  standardized  web  technologies 

Req.  10.  The  EAITS  database  and  web  server  integrations  shall  be  scalable  to  allow  the  use  of  existing 
technologies  (such  as  distributed  servers  and  cloud  computing)  for  deployment  of  the 
system  to  large  number  of  users 

3.3  Compute  Resource  Requirements 

Req.  11.  EAITS  shall  run  on  commodity  hardware 

Req.  12.  EAITS  shall  operate  on  both  the  Linux  or  Windows  operating  system 

Req.  13.  EAITS  shall  operate  in  a  standard  TCP/IP  network  environment  including  deployment  on 
Internet  accessible  webservers 

3.4  Documentation  Requirements 

Req.  14.  EAITS  shall  include  installation  and  maintenance  documentation 

Req.  14. a  Installation  and  maintenance  documentation  shall  be  written  to  enable 
computer  professionals  to  receive  a  delivery  of  EAITS  (e.g.,  on  a  CD-ROM),  and 
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set  up  the  demonstration  system  on  appropriate  hardware  and  software 
environment. 

Req.  15.  EAITS  shall  include  tutorials  for 

Req.  15. a  Students  on  the  use  of  the  EAITS  learning  environment 
Req.  15. b  Content  developers  on  the  use  of  authoring  tools 

Req.  16.  EAITS  shall  include  web  service  documentation  that  can  be  used  by  computer  professionals 
to  modify  the  integration  between  the  EAITS  learning  environment  and  corresponding  web 
services 

3.5  Interface  Requirements 

Req.  17.  The  EAITS  shall  have  a  class  design  (included  in  Req.  14)  that  provides  abstractions  for 
extension  of  components  (listed  in  Req.  8)  with  new  algorithms 

Req.  18.  EAITS  web  services  shall  be  well-defined  and  unambiguous 

Req.  19.  EAITS  web  services  shall  be  defined  in  the  EAITS  web  service  documentation 

Req.  20.  EAITS  shall  be  configurable  to  turn  off  specific  capabilities  which  can  be  used  to  perform 
ablation  assessments  and  analyses  of  the  system  learning  outcomes 

3.6  Security  Requirements 

Req.  21.  EAITS  shall  apply  best-practices  for  authenticated  access  to  the  learning  environment  and 
authoring  tools 

Req.  22.  EAITS  authoring  tools  shall  use  an  automatic  source  control  and  configuration  management 
system 

Req.  23.  The  EAITS  database  shall  separate  personally  identifiable  information  (Pll)  from  event  logs 
to  facilitate  anonymized  analysis  of  pedagogical  data 

3.7  User  Requirements 

Req.  24.  EAITS  shall  allow  participation  in  learning  activities  (listed  in  Req.  2)  by 
Req.  24. a  Students  for  pedagogical  purposes 
Req.  24. b  System  developers  for  demonstration  purposes 

Req.  25.  EAITS  shall  allow  development  of  knowledge  sources/content  by  subject  matter  experts 
such  as  educators  and  content  developers 

3.8  Performance  Requirements 

Req.  26.  EAITS  performance  requirements  are  to  be  determined  in  the  context  of  deployment 
scenarios  and  in  conjunction  with  representatives  from  the  Government  bodies  interested 
in  transition  the  system  to  real  users  within  their  organizations. 
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